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(57) Disclosed is a method, apparatus, and system 
for detecting human face in an image. The method com- 
prises: for a subset of pixels in said image, deriving a 
first variable from said gray-level distribution of said im- 
age; for said subset of pixels, deriving a second variable 
from a preset reference distribution, said reference dis- 
tribution being characteristic of said object; evaluating 
the correspondence between said first variable and said 
second variable over the subset of pixels; determining 
if said image contains said object based on the result of 
the evaluation step. 
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Description 

Field of the Invention 

5 [0001] The present invention relates to an innage processing method, apparatus, and system for determining the 

human face in an image, and a storage' medium. • 

Background of the Invention 

10 [0002] Image processing method for detecting or extracting a feature region of a given image is very useful. For 
example, it can be used to determine the human face(s) in a given image. It is very useful to determine the human 
faces in an image, especially in an image with a complex background. Such a method can be used in many fields, 
such as telecommunication conferences, person-to-machine interface, security checking-up, monitor system for track- 
ing human face, and image compression, etc. 

15 [0003] It is easy for a human being (an adult or a baby) to Identify human face in an image with a complex background. 
However, no efficient way has been found out to detect human face(s) in an image automatically and quickly. 
[0004] Determining whether a region or a sub-image in an image contains a human face is an Importing step in the 
human face detection. At present, there are many ways for detecting human face. For example, a human face can be 
detected by making use of some sa|ient features (such as two eyes, the mouth, the nose, etc.) and the Inherent geo- 

20 metric positional relations among the salient features, or making use of the symmetric characters of human face, 
complexion features of human face, template matching and neural network method, etc. For instance, a method is 
described in Haiyuan Wu, "Face Detection and Rotations Estimation using Color Information.", the 5th IEEE Interna- 
tional Workshop on Robot and Human Communication, 1996, pp341-346, in which a method is given for utilizing human 
face features (two eyes and the mouth) and relations among the features to detect human face. In this method, the 

25 image region to be determined is first studied to find out whether the needed human face features can be extracted. 
If yes. then the matching' degree of the extracted face human features to a known is human face model investigated, 
wherein the human face model describes the geometric relations among the human face features. If the matching 
degree is high, the image region is supposed to be an image of a human face. Otherwise, it is determined that the 
image region does not contains a human face. However, the method relies too much on the quality of the image to be 

30 investigated, and it is too much influenced by lighting conditions, the complexity of the image's background and the 
human race difference. Especially, it is very hard to determine human face exactly when the image quality is bad. 
[0005] There have been other prior art disclosures regarding hunhan face detection, such as: 

1 . "Region-Based Template Deformation And Masking For Eye-Feature Extraction And Description", JYH-YUAN 
35 DENG and PEIPEI LAI, Pattern Recognition, Vol. 30, No. 3. pp. 403-419.1997; 

2. "Generalized likelihood ratio-based face detection and extraction of mouth features", C. Kervrann, F. Davoine, 
P. Perez, R. Forchheimer, C. Labit, Pattern Recognition Letters 18 (1997)899-912; 

3. "Face Detection From Color Images Using a Fuzzy Pattern Matching Method**, Haiyuan Wu. Qian Chen, and 
Masahiko Yachida, IEEE Transactions On Pattern Analysis And Machine Intelligence, Vol. 21, No. 6. June 1999; 

40 4. "Human Face Detection In a Complex Background". Guangzheng Yang and Thomas S. Huang. Pattern Rec- 

ognition. Vol. 27. No.1. pp. 53-63. 1994; 

5. "A Fast Approach for Detecting Human faces in a Complex Background ", Kin-Man Lam, Proceedings of the 
1998 IEEE International. Symposium on Circuits and System, 1998. ISCAS*98 Vol. 4, pp85-88. 

45 Summary of the Invention 

[0006] It is an aim of the present invention to provide an improved image processing method, apparatus and storage 
medium for detecting objects in an image with a gray-level distribution. 

[0007] It is a further aim of the present invention to provide an improved image processing method, apparatus, and 
50 storage medium. Said image processing method and apparatus can detect an object In a given image quickly and 
effectively. 

[0008] In the present invention, a method for determining an object in an image having a gray-level distribution is • 
provided, which is characterized in that said method comprises the steps of: 

55 for a subset of pixels in the image, deriving a first variable from said gray-level distribution of said image. 

for said subset of pixels, deriving a second variable from a preset reference distribution, said reference distribution 
being characteristic of said object; 

evaluating the correspondence between said first variable and said second variable over the subset of pixels; 
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determining if said image contains said object based on the result of the evaluation step. 

[0009] Further, in the present invention, a method for determining an object in an image having a gray-level distri- 
bution is provided, said method comprises the steps of: 

5 

a) determining a sub-image in the image; 

b) selecting a subset of pixels based on the sub-image; 

c) deriving a first variable from said gray-level distribution of said image; 

d) for said subset of pixels, deriving a second variable from a preset reference distribution, said reference distri- 
10 bution being characteristic of said object; 

e) evaluating the correspondence between said first variable and said second variable over the subset of pixels; 

f) determining if said image contains said object based on the result of the evaluation step. 

[0010] In an embodiment object detection is carried out by evaluating two vector fields, so that adverse and uncertain 
15 effects such as non-uniform illumination can be eliminated, and the requirement on the quality of the image is lowered, 
while the varieties of images that can be processed with the method are increased. Moreover, gradient calculation is 
relatively simple, thus reducing the time needed for performing the detection. 

[0011] As a specific embodiment of the invention, a sub-image, in which a target object (such as a human face) is 
to be detected, is determined by detecting one or more characteristic features (such as a pair of dark areas which are 
20 expected to be human eyes) in the image, providing an effective way for detecting a predetermined object(s;) in the 
image. 

[0012] As a specific embodiment, the method limits the evaluation of correspondence between two vector fields in 
an area within the part of image to be detected, such as an annular area for human face detection, in which the cor- 
respondence between the two vector fields is more obvious, thus allowing the evaluation of the correspondence be- 
25 coming more effective while reducing the time needed for carrying out the detection. 

[0013] As a further embodiment, the method performs a weighted statistical process during the evaluation, allowing 
the pixels with larger gradient (i.e. more obvious characteristic feature) have greater contribution to the outcome of the 
evaluation, thereby making the detection more effective. 

[0014] As a further embodiment, the method uses both weighted statistical process and non-weighted statistical 
30 process and determine if an object is included in the image to be detected on the basis of the results of both the 
processes, thus increasing the accuracy of the detection. 

[0015] The present invention also provides an image processing method for detecting an object in ah image, conrv 
prising: 

35 setting a rectangular region in the image; 

setting an annular region surrounding the rectangular region; 
calculating the gradient of gray level at each pixel in said annular region; 
determining a reference gradient of each pixel in the annular region; and 

detemnining if said object is contained in said rectangle on the basis of the gradient of the gray level and the 
40 reference gradient at each pixel in said annular region. 

[0016] Further, the present invention also provides an image processing apparatus for determining a feature portion 
in an image, comprising: 

45 means for determining a rectangular region to be detected in the image; 

means for setting an annular region surround said rectangular region; 
means for calculating the gradient of gray level at each pixel in said annular region: 
means for calculating a reference gradient for each pixel in the annular region: and 

means for determining if said object is contained in said rectangular region on the basis of the gradient of the gray 
so level at each pixel and the reference gradient at each pixel in said annular region. 

[0017] Moreover, the present invention provides a carrier carrying program code for object-detection in ah image 
with a gray-level distribution. 

characterized in that said program code comprises: 

55 

codes for determining a sub-image in said image; 

codes for selecting a subset of the pixels in said sub-image; codes for deriving, for said subset of pixels, a first 
variable from said gray-level distribution of said image. 
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cx)des for deriving, for said subset of pixels, a second variable from a preset reference distribution, said reference 
distribution being characteristic of said object; 

codes for evaluating the correspondence between said first variable and said second variable over the subset of 
pixels; and 

5 codes for determining if said Image contains said object based on the result of the evaluation step. 

[0018] Moreover, the present invention provides a human eye detecting method for detecting human eyes in an 
image, comprising: 

10 a read step of reading the gray level of each pixel in the each column in the image; 

a segmenting step of segmenting each column into a plurality of intervals . and labeling each of the intervals as 
valley region , intermediate region or peak region; 

a merging step of merging the valley region of the each column and the valley region of its adjacent column, and 
generating an eye candidate region; and 
15 a determination step of determining the human eye from the eye candidate regions. 

[0019] Other objects and features will become apparent from the following embodiments and drawings. The same 
reference numeral in the drawings indicates the same or the like component. 

20 Brief Description on the Drawings 

[0020] The accompanying drawings, which are incorporated herein and constitute a part of the specification, illustrate 
embodiments of the invention and, together with the description, serve to explain* the invention. 

25 Fig. 1 is a block diagram showing an embodiment of the image processing system of the present invention. 

Fig. 2 is a block diagram showing an embodiment of an arrangement of human face detection apparatus according 
to the present invention. 

Fig. 3 schematically shows an original image to be detected. 

Fig. 4 is a flow chart showing a human face detecting process according to the first embodiment of the present 

•30 invention-. 

Fig. 5 is a diagram showing a sub-image (rectangular region) determined for detection in the image of Fig. 3 and 

the determined annular region around it. 

Fig. 6 is a diagram showing several pixels in an image. 

Fig. 7 is a diagram showing contour lines of a reference distribution for pixels in the annular region. 
35 Fig. 8A is a diagram showing another example of an original image to be detected. 

Fig. 8B is a diagram showing a sub-image (rectangular region) determined in the original image of Fig. 8A. 

Fig. 8C is a diagram showing a annular region determined for the rectangular region in Fig. 88. 

Fig. 9 is a flow chart showing the human face detection process of another embodiment according to the present 

invention. 

^0 Fig. 10 shows a reference distribution for use in detecting human face in an image; the reference distribution has ^ 

contour lines as shown in Fig. 7. 

Fig. 11 is for showing a way for generating a sub-image for human face detection from a pair of dark (eye) areas 
detected, which are expected to correspond a pair of human eyes. 

Fig. 12 Is a block diagram showing the arrangement of an eye detecting device according to an embodiment of 
45 the present invention; 

Fig. 13A is a flow chart showing the procedure of searching human eye areas. 

Fig. 138 is an example of an original image to be detected. 

Fig. 14A is a flow chart for segmenting every each column In an image. 

Fig. 148 Is an example for showing a column of pixels In an Image. 
50 Fig. 14C is an example for showing the gray level distribution of a column. 

Fig. 14D is a diagram showing the gray level of a column segmented into intervals. 

Fig. 14E is an example for showing a segmented column in an image. 

Fig. 14F is a diagram showing the determination of a segment point in a column. 

Fig. 15A is a flow chart showing the process for merging valley regions in the columns. 
55 Fig. 1 58 is a diagram showing the columns in an image and the valley' regions and the seed regions in each column. 

Fig. 15C is an image showing the detected candidate eye areas. 

Fig. 16A is the flow chart showing the process for determining eye areas in accordance with the present Invention. 
Fig. 168 is a diagram showing a candidate eye area and its circunvrectangle. 
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Fig. 16C is an image showing the detected eye areas. 

Fig. 17A Is a flow chart showing the process for adjusting segnnent border. 

Fig. 17B is a diagram showing the merger of a segment point to its adjacent intervals. 

Fig. 17C is a diagram showing the merger of an intermediate region to its adjacent vailey region. 

Fig. 18A is a flow chart showing a process forjudging whether a valley region can be merged into a seed region. 

Fig. 18B is a diagram showing a seed region's predicted valley region. 

Fig. 18C is a diagram showing an overlap between two valley regions. 



Description of the Preferred Embodiments 

10 

[0021] Preferred embodiments of the present invention will now be described in detail with reference to the accom- 
panying drawings. 

[0022] Fig. 1 is a block diagram showing an image processing system utilizing an image processing apparatus of 
the present invention. In the system, a printer 105, such as an ink-jet printer or the like, and a monitor 106 are connected 

15 to a host computer 100. 

[0023] Operating on the host computer 100 are an application software program 101 such as a word-processor, 
spreadsheet, Internet browser, and the like, an OS (Operating System) 102, a printer driver 1 03 for processing various 
drawing commands (image drawing command, text drawing command, graphics drawing command) for instructing the 
output of images, which are issued by the application software program 101 to the OS 102 for generating print data, 

20 and a monitor driver 104 for processing various drawing commands issued by the application software program 101 
and displaying data in the monitor 106. 

[0024] Reference numeral 112 denotes an instruction input device; and 113. its device driver. For example, a mouse 
that allows a user to point to and dick on various kinds of information displayed on the monitor 106 to Issue various, 
instructions to the OS 102 is connected. Note that other pointing devices, such as trackball, pen, touch panel, and the 
25 like, or a keyboard may be connected in place of the mouse. 

[0025] The host computer 100 comprises, as various kinds of hardware that can ran these software programs, a 
central processing unit (CPU) 108. a hard disk (HD) 107. a randonrvaccess memory (RAM) 109, a read-only memory 
(ROM) 110. and the like. 

[0026] An example of the face detection system shown in Fig, 1 may comprises Windows 98 available from Microsoft 
30 Corp. installed as an OS in a PC-AT compatible personal computer available from IBM Corp., desired application 
program(s) installed that can implement printing and a monitor, and a printer connected to the personal computer. 
[0027] In the host computer 100, each application software program 101 generates output image data using text 
data (such as characters or the like), graphics data, image data, and so forth. Upon printing out the output image data, 
the application software program 101 sends a print-out request to the OS 102. At this time, the application software 
35 program 101 issues a drawing command group that includes a graphics drawing command corresponding to graphics 
data and an image drawing command corresponding to image data to the OS 102. 

[0028] Upon receiving the output request from the application software program 101, the OS 102 issues a drawing 
command group to the printer driver 103 corresponding to an output printer. The printer driver 103 processes the print 
request and drawing commands inputted from the OS 102. generates print data for the printer 105, and transfers the 
40 print data to the printer 105: The printer driver 103 performs an image correction process for the drawing commands ' 
from OS 1 02, and then rasterizes the commands sequentially on a memory, isuch as a RGB 24-bit page memory. Upon 
completion of rasterization of alt the drawing command, the printer driver 103 converts the contents of the memory Into 
a data format with which the printer 105 can perform printing, e. g.. CMYK data, and transfers the converted data to 
the printer 105. 

45 [0029] Note that the host computer 1 00 can connect a digital camera 1 1 1 . that senses an object image and generates 
RGB image data, and can load and store the sensed image data in the HD 107. Note that the image data sensed by 
the digital camera 111 Is encoded, for example by JPEG. The sensed image data can be transfenred as image data to 
the printer 105 after it is decoded by the printer driver 103. 

[0030] The host computer 100 further comprises a face detection apparatus 114 for determining the human face In 
50 an image. The image data stored in HD 1 07 (or other memory) are read and processed by the face detection apparatus 
114. First, the possible positions of the human face region are determined and read, and whether the region contains 

one or more human faces is determined. Then, the portion(s) of the image that contains the determined human face 
(s) in the image can be sent to the printer 105 or monitor 106 under the control of OS 102. 

55 Face detection Apparatus 

[0031] Fig. 2 is a block diagram showing the arrangement of the face detection apparatus according to the present 
invention. 
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[0032] The face detection apparatus 114 of the present embodiment comprises a reading means 210, an eye area 
detecting device 218, a sub-image determining device 219, an annular region setting rneans 211. a first calculating 
means 212, a second calculating means 213, and a determination means 214. In the face detection apparatus 114, a 
reading means 210 executes an image reading process. The gray level of each pixel of an image stored in the HD 107 
5 or the RAM 109 or the like is read by the reading means 210. 

Human Face Detection Process 

[0033] Fig. 3 schematically shows an example of an original image to be detected. The original image contains a 
10 human face. The original image 300 can be input to the human face detection system by a digital device 111 such as 

a digital camera, a scanner or the like, and the original image is stored in the HD 107 or the RAM 109, or the like. 

[0034] Referring to Fig. 3, it can be seen that the shape of the human face contour in the original image 300 is 

generally closed to an ellipse, which has' nothing to do with human race, complexion, age and gender. Along the human 

face contour, a circumscribed rectangular region 300A can be drawn. 
15 [0035] Fig. 4 is a flow chart for showing a human face detection process according to an enribodiment of the present 

Invention. 

[0036] Referring to the flow charts in Fig. 4 and Fig. 3, an explanation to human face detection process forthe original 
image will be given. 

20 Reading Original Image and Determining Sut>-lmage (Rectangular region) to be Detected 

[0037] Referring to Fig. 4, the human face detection process starts in step S40. In step S41, reading means 210 
reads an original image 300 to be detected, and acquires the gray level of each pixel of the original image 300. If the 
original image 300 is encoded by, e.g., JPEG, the reading nneans 210 must first decode it before reads its image data. 

25 In step S41 , the sub-image determining device 21 9 determines one or more sub-images (or regions) 300A in the original 
image 300 for human face detection, and it also determines the location of the sub-image 300A in.the original image 
300. The sub-image 300A can be substantially rectangular and is the candidate region of a human face image portion. 
However, the shape of the sub-image is not limited to rectangular but can be any other suitable shape; 
[0038] A method and device, in accordance with the present invention, for determininig sub-image(s) 300A in an 

30 image for detection are described below. It is to be noted, however, that the manner of eye area detection for determining 
the sub-image is not limited to the method of the present invention as described below. Instead, . other methods and/ 
or processes, as those known in the art, can be utilized for determining the sub-image. 

Determination of Sub-image 300A 

35 

Eye detecting Device 

[0039] Fig. 12 is a block diagram showing the arrangement of the eye-detecting device according to the an embod- 
iment of the present invention; 

40 [0040] The eye detecting device 218 of the present embodiment comprises a segment means 1201, a mergei" means 
1202 and a determination means 1203. Referring to Figs. 14D and 14E, on the basis of the gray level of each pixel in 
a column C41 of an image, the column C41 of an image is segmented into a plurality of intervals 11-1, 11-2, ... 11-9, 
11-10 by segment means 1201. These intervals 11-1, 11-2, ... 11-9, 11-10 can be classified into three types: peak regions, 
valley regions and intermediate regions, according to the average gray level of the pixels in them. The terms "peak 

45 region, valley region and intermediate region" will be defined in detail later. Then, the valley regions of column C41 
can be obtained. In the same way, the segment means 1201 also divides other columns of the image into the three 
types and obtains their valley regions respectively. After ail the columns of an image have been niarked as the three 
types and their valley regions have been obtained, the merger means 1 202 executes the merging process and merges 
the valley regions in the adjacent columns. The merged valley regions are set as the human eye candidates. Then the 

50 human eye can be determined by determination means 1203. 

Detecting the eye areas 

[0041] A human eye detecting process for an original image will be explained below with reference to the flow chart 
55 in Fig. 13A. Fig. 13B is an example of an original Image to be detected. Assume that the original Image Is stored in a 
predetermined area in the HD 107 or the RAM 109, or the like. 

[0042] Referring to Fig. 13A, In step SI 32, each column of the original image is segmented into many intervals by 
the segment means 1201. With reference to Fig. 14E. the length of each of the intervals 11-1, 11-2, ... 11-9 and 11-10 
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is variable. For example, the length of Interval 11-1 is not equal to the length of interval 11-2. Some of the segmented 
intervals are marked as the valley regions on the basis of their average gray levels of pixels. In step SI 33, the valley 
regions in the adjacent columns are merged by the merger means 1202 to generate the eye candidate regions. Since 
the valley regions in each column have different lengths, the sizes of the eye candidate regions are also different from 
5 each other. In step SI 34, the human eye areas in the eye candidate regions are determined by the determination 
means 1203. Thus, areas corresponding to human eyes In the image can be detected. 

Segmenting Each Column of an Image 

10 [0043] Fig. 14A Is the flow chart showing the process for segmenting each column in an image In step SI 32, 
[0044] The terms " valley region", "peak region" and "intermediate region" are defined as below. 
[0045] Fig. 14B is an example for showing a column in the image. Referring to Fig. 14B, a column C41 of the original 
Image is read by the reading means 200. Fig. 14C shows a gray level distribution of the column C41. Fig. 14D Is a 
gray level distribution of the column segmented into intervals. In Fig. 14D, the reference numerals 11-5, 11-6.11-9 denote 

15 the segmented Intervals, respectively, and the gray level of each of the segments or Intervals Is the average of the 
gray levels of pixels in the same segment or interval in Fig. 14C. 

[0046] Fig. 14E is the segmented column of the image in Fig. 14B. Referring to Fig. 14E, the image data of a column 
C41 in the image is read by reading means 1200. For the image of Fig. 14B, the column C41 is segmented into 10 
intervals 11-1, 11-2,... 11-9 and 11-10. An interval's size Is the number of the pixels in the inten/al. For example. If the 

20 interval 11-2 comprises 12 pixels, the Interval 11-2's size is 12. 

[0047] With reference to Fig. 14D and 14E, If an interval's gray level is less than both of its adjacent intervals' gray 
level, then the interval is called a valley region. If an interval's (average) gray level is bigger than both of its adjacent 
intervals' gray level, the interval is called a peak region. On the other hand, if an interval's gray level is between its 
adjacent interval's gray levels, such an Interval Is called an intermediate region. As to column C41 of the embodiment. 

25 the gray levels of Intervals from 11-1 to 11-10 are 196, 189. 190. 185. 201. 194. 213. 178. 188. and 231 respectively. 
As to interval 11-6, its gray level Is 194, and the gray levels of its adjacent Intervals 11-5 and 11-7 are 201 and 213 
respectively. Since the gray level of interval 11-6 is less than that of its adjacent intervals 11-5 and 11-7, the interval 11-6 
Is determined as a valley region. In the same way, intervals 11-2, 11-4 and 11-8 are also determined as valley regions. 
As to interval 11-5, Its gray level Is 201. and the gray levels of Its adjacent Intervals are 185 and 194 respectively. Since 

30 the gray level of Interval 11-5 is bigger than that of its adjacent Intervals 11-6 and 11-7, the Interval 11-5 is determined 
as a peak region. In the same way, intervals 11-1, 11-3, 11-7 and 11-10 are also determined as peak regions. Further, 
as to interval. 11-9, its gray level is 188, the gray levels of its adjacent 11-8 and 11-10 are 178 and 231. Since the gray 
level of interval 11-9 is between the gray levels of Its adjacent intervals 11-8 and 11-10, interval 11-9 is determined as 
an intermediate region. 

35 [0048] As a valley region is also an interval, the ways for computing the valley region's gray level and size are the 
same as those for computing the Interval's gray level and size. It is also applied to the computation of the gray level 
and size of a peak region or an intermediate region. 

[0049] The process for segmenting every column in an Image in step SI 32 will be explained below with reference 
to Fig. 14A. 

40 [0050] Referring to Fig. 14A. the gray level of each pixel in the first column from the left of the detected image are 
read out in step S141. Inorderto segment the column Into Intervals of the three types, i.e.. valley regions, peak regions 
and intermediate regions, the segmenting points have to be determined. 

[0051] In step S142, whether a pixel in the column Is a segment point can be determined according to the values of 
first and second-order derivatives of the gray-level distribution at the pixel. Fig. 14F Is a diagram for showing the 
45 procedure to determine whether a pixel Is a segmenting point In a column. With reference to Fig. 14F. two adjacent 
pixels Pil and Pi2 are given In a column. 

[0052] Then, using any discrete derivative operator, the first and second-order derivatives at these two pixels Pil, 
PI2 are calculated. Assuming that the values of the first-order derivative at pixels PI1 and Pi2 are represented by Dlf 
and D2f respectively, and the values of the second-order derivative at pixels PI1 and PI2 are represented by D1s and 
50 D2s respectively, then if either of the following two conditions Is true: 

(D1s>0) and(D2s<0): 
{D1s<0)and(D2s^0) 

and either of the absolute values of Dlf and D2f is bigger than a predetemnined value - which is in the range of 6-15 
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but is preferably 8. then the pixel Pi1 is determined as a segmenting point. OthenA^rse the pixel Pil is not determined 

as a segmenting point. 

[0053] Thus, the segmenting points s11 , s12, ... s19 can be obtained in step S142. 

[0054] After the segmenting points in a column have been detemnined. the column can be segmented into a plurality 
of intervals in step S143. Then, in step S144, the intervals are divided into valley region, peak region and intermediate 
region in accordance with the gray levels of the intervals. The border of intervals is adjusted in step S146. The detail 
of step S145 will be described using detailed flow chart. In step 146, it is checked if all columns in the detected image 
have been segmented. If the column being segmented is not the last column of the detected image, the flow goes to 
step S147. In step S147, the gray levels of pixels in the next column are read out. Then the flow returns to step S142 
to repeat the process in step S142 and the subsequent steps. However, if the column being segmented is the last 
column of the detected image in step 146, i.e. all columns have been segmented, the flow ends in step S148. 
[0055] Alternatively, the above-mentioned segmenting process may start from the first column from the right of the 
detected image. 

Merging Valley Regions to Generate Eye Candidate Regions 

[0056] Fig. ISA is the flow chart for showing the process of merging valley region in the columns in step S133 in Fig. 
13A. Fig. 1 SB is a diagram showing the columns of an image and the valley regions and seed regions in each column 
of the image. In Fig. 15B, an image has h columns Coll, Col2. ... Coin. 

[0057] With reference to Fig. 15A and 15B, all the valley regions SI, S2, S3 and S4 in the first column CoM (most 
left) of the detected image are set as seed regions in step S151, A seed region is an aggregation of one or more valley 
regions. Since the gray level of a valley region is less than that of a peak region or a intermediate region, a seed region 
is usually a dark area in a column. 

[0058] In step 81 52 of Fig. 1 5A, the first valley region V2-1 in the next column Col2 is read out. Then the flow advances 
to step SI 53. In step SI 53, the first seed region SI is read out. In step SI 54, it is checked if the valley region V2-1 of 
column Col2 can be merged Into the seed region SI on the basis of the valley region V2-1 and the seed region S1 . If 
the valley region V2-1 of the valley region V2-1 can be merged into the seed region SI, then the flow goes to step 
S156 and the valley region V2-1 is merged into the seed region, then the valley region becomes a part of the seed 
region. However, if it is checked in step S1 54 that the valley region V2-1 can not be merged into the seed region S1 , 
the flow goes to step S155. In the present example, valley region V2-1 of column Col2 can not be merged into seed 
region SI. The flow advances to step SI 55. In step S155, it is checked whether the seed region is the last seed region. 
If the seed region is not the last seed region, then next seed region is read out in step S1S7 and the flow returns to 
step SI 54 to repeat the processes in step SI 54 and the subsequent steps. In the present exanriple, seed region SI is 
not the last seed region, so in step S157 the next seed region S2 is read out, and the steps of S154 and SI 55 are 
repeated. If it is checked In step SI 55 that the seed region is the last seed region (for example, the seed region S4 as 
shown in Fig SB), then the flow advances to step SI 58 and set the valley region that can not be merged into a seed 
region as a new seed region. Referring to Fig. 15B, since valley region V2-1 of column Col2 can not be nrierged into 
seed regions SI . S2. S3 or S4, that is, it is a valley region that can not be merged into any existing seed region, then 
valley region V2-1 of column Col2 is set as a new seed region in step S158. 

[0059] In step S159, it is checked if all the valley regions in the column Col2 have been processed' If all the valley 
regions in the column Col2 have been processed, the flow goes to step S1511. In step S1511, it is checked if all the 
columns have been processed. If the column is not the last column of the detected image, then the flow returns to step 
SI 52 to repeat the processes in step SI 54 and the subsequent steps. As column Col2 is not the last colunrin of the 
detected image, the flow returns to step S152. If all the columns have been processed, i.e., if the column is the last 
column Coin, the flow advances to step S1520. In step S1 520, all the seed regions are set as eye candidate regions. 
Then the flow ends in step SI 521. Fig. 15C is an example showing the result for merging valley regions to generate 
eye candidate regions in columns in a detected image in step S1 33. 

Determining Eye Areas 

[0060] Fig. 16A is the flow chart showing the process for determining eye areas in step S134. 

[0061] With reference to Fig. 16A, the first eye candidate region is read out in step S161 . Then, the flow advances 

to step S1 62. In step SI 62, the gray level of an eye candidate region is calculated. As described above, an eye candidate 

region comprises one or more valley regions. If an eye candidate region is comprised of n valley regions, i.e. valley 

region 1 . valley region 2, ... valley region n, then the eye candidate region's gray level calculated in step S162 is given 

by: 
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EyeGray1=(Valley1Grayl x pixels 1 +Valley2Gray1 x pixels 2... 

+ValleynGray 1 x pixels n)/Total Pixels (1) 

5 

where 

EyeGrayl is an eye candidate region's gray level; 

Valleyl Gray1 is the gray level of valley region 1 , pixels 1 is the number of pixels in valley region 1; 
10 Valley2Gray1 is the gray level of valley region 2, pixels2 is the number* of pixels in valley region 2; 

ValleynGrayl is the gray level of valley region n, pixels n is the number of pixels in valley region n; 
Total Pixels is the total number of the pixels included in an eye candidate region. 

[0062] Therefore, if an eye candidate region comprises 3 valley regions with gray levels of 1 2, 20 and 30 respectively, 
15 and each of the valley regions has 5, 6, and 4 pixels respectively, then the gray level of the eye candidate region will 
be (12 x 5 + 20 x 6 + 30 X 4)/15=20. 

[0063] Referring to step S162 of Fig. 16A, the gray level of an eye candidate region is calculated. If the eye candidate 
region's gray level is not less than a first threshold (for example, 160), the flow goes to step SI 610. In the present 
embodiment, the first threshold is within the range of 100 to 200. In step SI 6 10, the eye candidate region is determined 

20 as a false eye area and is rejected. Then the flow goes to step SI 68. In step S168, it is checked if alt the eye candidate 
regions in the detected image have been processed. If it is not the last eye candidate region, then the next eye candidate 
region will be read in step S169. then the flow returns to step S162 to repeat the processes in step SI 62 and the 
subsequent steps. However, if it is checked in step S168 that the eye candidate region is the last eye candidate region, 
then all eye candidate regions in the detected image have been determined, and the flow ends in step SI 611 . 

25 [0064] At step S162, if the gray level of the eye candidate region is less than the first threshold, the flow advances 
tostepS163. 

[0065] The background gray level of the eye candidate region is calculated in step SI 63. The background gray levels 
of valley regions included in the eye candidate region determine the background gray levels of an eye candidate region. 
A valley region's background gray level is the average gray level of its adjacent intervals* gray levels. The eye candidate 
30 region's background gray level calculated in step 8163 is given by : 

EyeBGrayl = (Valleyl BGrayl +Valley2BGray1 ...+ValleynBGray 1)/n; (2) 

35 where 

EyeBGaryl is an eye candidate region's background gray level; 
Valleyl BGrayl is the background gray level of valley region 1; 
Valiey2BGrayl is the background gray level of valley region 2... 
40 ValleynBGrayl is the background gray level of valley region n; and 

n is the number of valley regions included in an eye candidate region. 

[0066] At step S163. the background gray level of an eye candidate region is calculated. If the eye candidate region's 
background gray level is not bigger than a second threshold(for example, 30) in step S 1 63, the flow goes to step SI 61 0. 
45 In the present embodiment, the second threshold is within the range of 20 to 80. For the present example, in step 
S1 61 0, the eye candidate region is determined as a false eye area and is rejected. Then the flow goes to step S1 68. 
[0067] At step S163, if the background gray level of the eye candidate region is bigger than the second threshold, 
the flow advances to step SI 64. 

[0068] The difference of background gray level of the eye candidate region and the gray level of the eye candidate 
50 region is calculated in step S164. If the difference is not bigger than a third threshold( for example. 20). the flow goes 
to step S1610. In the present embodiment, the third threshold is withih the range of 5 to 120. In step 81610, the eye 
candidate region is determined as a false eye area and is rejected. Then the flow goes to step S168. 
[0069] At step 8163, if the difference of background gray level of the eye candidate region and gray level of the eye 
candidate region is bigger than the third threshold, the flow advances to step 8165. 
55 [0070] The ratio of the width to the height of an eye candidate region is calculated in step SI 65. 

[0071] As to the height and the width of an eye candidate region, we have the following definitions. Valley region's 
size is the number of pixels included in a valley region. For example, if a valley region comprises 5 pixels, then the 
valley region's size is 5. The size of an eye candidate region is the sum of the sizes of the valley regions included in 
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the eye candidate region. The width of an eye candidate region is the number of valley regions included in the eye 
candidate region. The height Hd of an eye candidate region is given by: 

Hd = Sd/Wd (3) 

[0072] Where. 

Hd is the height of an eye candidate region; 
Sd is the size of an eye candidate region; 
Wd is the width of an eye candidate region. 

[0073] With reference to step S165 in Fig. 16A, the ratio of the width to the height of an eye candidate region is 
calculated. If in step S165 the ratio of the width to the height of an eye candidate region is not bigger than a fourth 
threshold(for exannple, 3.33), the flow goes to step S1610. In the present embodiment, the fourth threshold is within 
the range of 1 to 5. In step SI 610, the eye candidate region is determined as a false eye area and is rejected. Then 
the. flow goes to step S168. 

[0074] At step S165, if the ratio of the width to the height of an eye candidate region is bigger than the fourth threshold, 
the flow advances to step S1 66. 

[0075] The ratio of the size of an eye candidate region to that of its circum-rectangle is calculated in step S166. Fig. 
16B is a diagram showing an eye candidate region and its circum-rectangle. With reference to Fig; 16B. an eye can- 
didate region D1 and its circum-rectangle DC1 are given. As seen from Fig. 16B, the eye candidate region's circum- 
rectangle DC1 is the smallest rectangle that encircles the eye candidate region D1. The size of an eye candidate 
region's circum-rectangle is the number of pixels included in the circunl-rectangle. The size of ian eye candidate region 
is the number of pixels included in the eye candidate region. 

[0076] At step S166, the ratio of the size of an eye candidate region to that of its circum-rectangle is calculated. If 
the ratio is not bigger than a fifth threshold (for example 0.4) in step S166, the flow goies to step SI 610. In the present 
embodiment, the fifth threshold is within the range of 0.2 to 1. In step S1610. the eye candidate region is determined 
as a false eye area and is rejected. Then the flow goes to step S168. 

[0077] At step S166, if the ratio of the size of an eye candidate region to that of its circum-rectangle is bigger than 
the fifth threshold, the flow advances to step S167, where the eye candidate region is determined as a true eye area. 
[0078] After step 8167, the flow advances to step SI 68 and determines whether the eye candidate region is the last 
eye candidate region. If NO, then the next eye candidate region is read in step SI 69 and the flow returns to step SI 62. 
If YES in step SI 68, then all eye areas are determined. Fig. 16C is an example showing the result for detecting eye 
areas in an image in step SI 33. 

Adjusting Segment's Border 

[0079] Fig. 17A is a flow chart showing the process of adjusting segment border in step S145 in Fig. 14Av 
[0080] With reference to Fig. 17A, the gray level of a segment point is compared with the gray levels of its two ' 
adjacent intervals, then the point is merged into the interval whose gray level is closer to the point's gray'level in step 
S171. For example, referring to Fig. 17B, the gray level of segment point S is 80, Its adjacent intervals are intervals I nl 
and In2. The gray levels of intervals In1 and In2 are 70 and 100 respectively. Since the gray level of interval In 1 is 
closer to that of point S, then S is merged into intervallnl . % 
[0081] Further, the flow advances to step S172. In step S172, the first intermediate region is read out. Then the gray 
levels of the intermediate region and its adjacent valley region and peak region are calculated in step S173. After the 
gray levels of them have been calculated, the flow advances to step S174. In step S174, a comparison is made to 
decide whether GR is less than GPxTh6+GVx(1-Th6). wherein. GR denotes the gray level of the internriediate region. 
GV denotes the gray level of the intermediate region's adjacent valley region, GP denotes the gray leverol the inter- 
mediate region's adjacent peak region. Th6 is the sixth threshold(for example" 0.2). In the present embodiment, the 
sixth threshold is within the range of 0 to 0.5. If the decision of step SI 74 is NO, the flow goes to step SI 76. If the 
decision of step S174 is YES, then the intermediate region is merged into the valley region in step S175. 
[0082] Fig. 17C is a diagram showing an example for merging the intermediate region to the adjacent valley region. 
X axis shown in Fig. 17C represents the position of each column, Y axis shown in Fig. 17C represents the gray level 
of each region. 

[0083] Referring to Fig. 17C, intermediate region Rel's gray level is 25, valley region Val's gray level is 20, and 
peak region Pel's gray level is 70. When the sixth threshold is set as 0.2, then 
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GP X Th6+GVx(1-Th6) = 70 x 0.2+20 x 0.8=30 > GR=25 

[0084] Therefore, the decision In step S174 is YES, so the intermediate region Re1 will be nnerged into the valley 
5 region Va1. Further, internnediate region Re2's gray level is 40, peak region Pe2's gray level is 60, then 

GPxTh6+GVx(1-Th6) = 60x0.2+20x0.8=28 < GR = 40; 

10 [0085] Therefore, the decision in step S174 is NO, so intermediate region Re2 will not be merged into the valley 
region Va1. 

[0086] Referring to step S176 in Fig. 17A, it is checked if all the intermediate' regions in the detected image have 
been processed. If the intermediate region is not the last Intermediate region, the next Intermediate region will be read 
In step S177. then the flow returns to step S173 to repeat the processes In step S173 and the subsequent steps. 
15 However, if it is checked in step S176 that the intermediate region is the last intermediate region, i.e. the processing 
for all Intermediate regions is complete, the flow ends in step S178. Thus, all segment borders in the detected Image 
have been adjusted. 

Determining Whether a Valley Region Can be Merged Into a Seed Region 

20 

[0087] Fig. 18A Is the flow chart for showing the process for determining whether a valley region can be merged into 
a seed region in step S1 54 in Fig. 15A. 

[0088] Fig. 18B is a diagram showing a seed region's predicted valley region. A seed region's predicted valley region 
Isn't a real existing valley region in any columns of the detected Image. It is a valley region that Is assumed In the next 

25 column of the column including the most adjacent valley region at the right of the seed region, and its position is the 
same as that of the most adjacent valley region at the right of the seed region. With reference to Fig. 18B, valley region 
Va3 Is the most adjacent valley region at the right of seed region Se1. Valley region Va3 Is in the column Col1, and 
column Col2 is the next column of column Col1 . Then valley region Va1 is the predicted valley region of seed region 
Sel . This predicted valley region is in column Col2, and Its position is the same as that of valley region VaS but in a 

30 different column. 

[0089] F\g. 18C is a diagram showing an overlap region of two valley regions. The overlap region of two valley regions 
is an area in which the pixels belong to the two valley regions. 

[0090] Referring to Fig. 18C, the interval from point B to point D is a valley region Va1, the Interval from point A to 
point C is a valley region Va2. the valley region Va1 is the predicted valley region of the seed region Sel , the valley 
35 region Va2 is a real valley region in column Col2. Then, the interval from point B to point C is the overlap region of the 
valley region Va1 and the valley region Va2. 

[0091] The procedure for judging whether a valley region can be merged Into a seed region will be explained below 
with reference to the flow chart in Fig. 18A. Referring to Fig. 18A, the overlap region of a valley region and a seed 
region's predicted valley region is calculated in step S181. 

40 [0092] After the overlap region has been calculated, the flow advances to step S182. In step S182, a comparison is 
made to decide whether Osize/Max(Vslze, SVsIze) is bigger than Th7, wherein. Osize is the size of overlap of the 
valley region and the seed region's predicted valley region, Max (Vsize, SVsIze) is the maximum of the size of the 
valley region and that of the seed region's predicted valley region, and Th7 is the seventh threshold (for example, 0.37). 
The seventh threshold is within the range of 0.2 to 0.75. 

45 [0093] If the decision of step S1 82 is NO, the flow goes to step S1 88; then the valley region can not be merged into 
the seed region, and the flow ends in step S189. OthenA^ise. if the decision of step S182 is YES, then the flow advances 
to.stepS183. 

[0094] In step S183. the gray levels of the valley region and the seed region are calculated. Then the flow advances 
. to step S184. In step S184. it is detemnined whether |GValley-GSeed| is less than Th8. wherein GValley is the gray 
50 level of the valley region. GSeed Is the gray level of the seed region, and Th8 is an eighth threshold(for example, 40). 
, The eighth threshold is within the range of 0 to 60. If the decision of step S184 is NO. the flow goes to step S188; then 

the valley region can not be merged Into the seed region, and the flow ends in step SI 89. Otherwise, if the decision 

of step S 184 is YES. then the flow advances to step S185.. 

[OOisS] In step S185, the luminance values of the valley region's background, the seed region's background, the 
55 valley region and the seed region are calculated. 

[0096] As to the luminance value of a pixel In an image, it can be calculated by: 
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G=0.1 221 9 X L-0.0009063 x +0.000036833526 x L^- 
0.0000001 267023 x L^* +0.0000000001 98783 X L^: (4) 

5 

where G is gray level of a pixel ranged from 0 to 255; and L is luminance level of a pixel and is also ranged from 
0 to 255. 

[0097] Therefore, the luminance value of a pixel can be obtained from its gray level In an innage. On the other hand, 
the gray level of a pixel can be obtained from Its luminance value. 

[0098] For the present example, the pixels Pi1 and Pi2 in Fig. 14 F have the gray levels of 50 and 150 respectively. 
With formula (4), it can be determined that the luminance valves of pixels Pi1 and Pi2 are 128 and 206 respectively. 
[0099] Referring to Fig. 18A, after step SI 85, the flow advances to step SI 86. In step SI 86, it is determined whether 
Min((Lvb-Lv), (Lsb-Ls))/Max((Lvb-Lv), (Lsb-Ls)) is bigger than Th9, wherein Lv is the luminance value of the valley 
region, Ls is the luminance value of the seed region, Lvb is the luminance value of the valley region's background, Lsb 
is the luminance value of the seed region's background. Min {(Lvb-Lv), (Lsb-Ls)) is the minimum of (Lvb-Lv) and (Lsb- 
Ls), Max ((Lvb-Lv), (Lsb-Ls)) is the maximum of (Lvb-Lv) and (Lsb-Ls), and Th9 is a ninth threshold (for example, 0.58). 
The ninth threshold is within the range of 0.3 to 1 . If the decision of step SI 86 is NO, the flow goes to step SI 88; then 
the valley region can not be merged Into the seed region, and the flow ends in step SI 89. Otherwise, if the decision 
of step S186 is YES, the flow advances to step S187. 

[0100] In step S187, the valley region is merged into the seed region, then the flow ends in step SI 89. 
[0101] As can be seen from the above, the present invention provides a method and a device for quickly detecting 
areas, each of which corresponds to a human eye, in an image with a complex background, without requiring that 
detected image has a very high quality, thus greatly reducing the possibility that human eyes in the image fails to be 
detected. The method of the invention allows for the precision detection of human eyes under different scales, orien- 
tation and lighting condition. Therefore, with the method and device of the present invention, the human eyes In an 
image can be quickly and effectively detected. 

[0102] The method and device of the present invention explained above are described with reference to detection 
human eyes in an image, however, the method and device of the present invention are hot lihriited to detection human 
eyes, it is applicable to other detection method, for example, method to detect flaw portion on a circuit board. 
^ [0103] In addition, the above method and device for detecting areas corresponding to human eyes in an Image are 
used for detection of human face contained in an image, but the method and apparatus for the detection of human 
face in an image of the present invention may also use other methods and devices'for detection of human eyes in the 
Image on which detection is to be performed, for example, the method disclosed in Kin-Man Lam. "A Fast Approach 
for Detecting Human faces In a Cohiplex Background Proceedings of the 1998 IEEE International Symposium on 
Circuits and System, 1 998. ISCAS'98 Vol. 4, pp85-88 can be used in the method and apparatus of the present invention 
for human face detection. 

[0104] Upon detecting at least two eye (or dark) areas, an arbitrary pair of the eye areas is selected as a pair of 
candidate eyes. For each pair of selected eye areas, the distance L between the centers of them is determined. Then, 
a rectangular sub-Image 300A is determined as show in Fig. 11 . For human face detection, in addition to the rectangular 
sub-image as shown in Figure 11 , another rectangular sub-image is also determined for detection of human face, which 
is symmetrical to the rectangular siub-image of Figure 11 with respect to the lines passing the centers of the pair of eye 
areas. 

[0105] It Is to be noted that the values/ratios shown In Fig. 11 are not necessarily strict, rather, values/ratios varying 
within certain range (such as ±20 or so) from those as shown in Fig. 11 are acceptable for performing the method for 
human face detection of the present invention. 

[0106] In the case that more than two eye areas are detected, each possible pair of eye ai-eas are chosen for deter- 
mination of respective sub-images. And for each pair of eye areas in the image, two rectangular sub-images may be 
determined in the manner as described above. 

[0107] Then, for each sub^image thus determined, detection is performed as to whether the sub-Image corresponds 
^ to a picture of human face, as described below. 

[0108] Moreover, the shape of the sub-image 300A for detection is not linriited to rectangular, rather, it can be any 
suitable shape, e.g. ellipse etc. 

[0109] Also, the location of the human face portion (sub-image) of the image can be determined not only by detecting 
eye areas (dark areas) in the image but also by detecting other features of human face such as mouth, nose, eyebrows, 
etc. in the image. Moreover, for determining the location of human face in an image by detecting human face features 
such as eyes, mouth, nose, eyebrows, etc. in the image, in addition to the method of human eye detection of the present 
invention as disclosed above, any suitable prior art methods for determining human face features (eyes, eyebrows. 
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nose, mouth, etc.) can be used for locating the sub-innagefor human face detection, including the methods as disclosed 
in the prior art references listed in the Background of the Invention of the present specification. 

Determination of Annular region 

5 

[0110] Then, the flow goes to step S42. In step S42, an annular region 300R (Fig. 5) surrounding the sub-image 
(rectangular region) 300A is determined by an annular region determination means 211 . The generation of the annular 
region 300R will be described in detail with reference to Fig. 5. 

[0111] Fig. 5 is a diagram showing the rectangular region SODA of Fig. 3 and the determined annular region 300R 
10 around it. In the Cartesian coordinate system shown in Fig. 5. the upper left corner of the original image 300 is taken 
as the coordinate origin, wherein X and Y axes extend in horizontal or vertical directions respectively. 
[0112] Refemng to Fig. 5, by making use of reading means 210 as shown in Fig. 2, the pre-stored original image 
300, as shown in Fig. 3, is read from the HD 107 or the RAM 109, and the location or rectangular region 300A relative 
to the original Image 300 is also acquired. In the present embodiment, the coordinates of each of the four corners of 
15 rectangular region 300 are (230. 212). (370. 212), (230, 387) and (370. 387) respectively in the given Cartesian coor- 
dinate system, thus its width and length is 140 and 175 respectively 

[0113] From rectangle 300A. two rectangles 300A1 and 300A2 are derived, Rectangle 300A1 is linearly enlarged 
with respect to rectangle 300A with a factor of 9/7, and rectangle 300A2 is reduced with respect to rectangle 300A with 
a factor of 5/7. Two rectangular regions 300A1 and 300A2 are produced. Thus the coordinate of each of the four corners 

20 of the first rectangular region 300A1 in this coordinate system is (210. 187), (390. 187), (210, 412) and (390, 412) 
respectively; while the coordinate of each of the four corners of the second rectangular region 300A2 in this coordinate 
system is (250, 237), (350, 237), (250, 362) and (370. 362) respectively. The region between the first rectangular region 
300A1 and the second 300A2 forms the annular region 300R for the above-mentioned rectangular region 300A. 
[0114] The annular region 300R shown in Fig. 5 is produced by first enlarging the width and length of the rectangular 

25 region 300A with a factor of 9/7 and then reducing that with a factor of 5/7. But the present invention is not limited to 
it, as the width and length of the larger rectangular region can be at a factor of n and those of the smaller rectangle at 
a factor of m. thus the larger rectangular region and the smaller 300A2 rectangular region are respectively produced, 
where m within the range of 0 to 1 and n within the range of 1 to 2. 

30 Calculating the Gradient of the Gray Level at Each Pixel in the Annular region and its Weight 

[0115] Returning to Fig. 4, the flow goes to step S43 after step S42. In step S43. the gradient of the grayHevel at 
each pixel in the annular region 300R is calculated by a first calculating means 212, while the weight of the gradient 
of gray level at each pixel in the annular region 300R is also determined. 
35 [0116] With reference to Fig. 5 and Fig. 6, how the human face detection apparatus according to the present invention 
determines the gradient and its corresponding weight of the gray level at each pixel in the annular region 300R are 
explained. 

[0117] In general, each pixel in an image is surrounded by many other pixels. Hence, the gradient of the gray level 
of a pixel can be determined by the gray levels of k^ pixels around the certain pixel. Where, k is an integer ranged from 
40 2 to 15. In the present embodiment, k is given to be 3. In the present invention, the Sobel operator is used to calculate 
the gradient of the gray level distribution at each pixel In the annular region 300R in the original image. 
[0118] Referring to Fig. 6, how to determine the gradient of the gray level of each pixel in the annular region 300R 
is explained, wherein the coordinates of pixel P In the coordinate system is (380, 250). 

[0119] Refemng to Fig. 6. the neighboring pixels of pixel P. pixels PI , P2, P3, P4, P5, P6, P7 and P8 are selected, 

45 SO the surrounding pixels of pixel P are used. In the present embodiment, the gray level of each pixel in the Image Is 
from 0 to 255. Wherein the gray level of pixel P is 122. while the gray level of pixels PI. P2, P3. P4, P5, P6, P7 and 
P8 are 136. 124. 119. 130. 125, 132. 124, and 120 respectively 

[0120] According to Sobel operator, with the gray levels of the neighbor pixels PI. P2, ... P8, etc., the gradient of 
gray level of pixel P Is given by formula (5): 

50 

DX1 =(G3+2 X G5+G8)-(G1 +2 X G4+G6) 

D Y1 = (G6+2 X G7+G8)-(G1 +2 x G2+G3) (5) 

55 

[0121] Where DX1 is the x component of the gradient of the gray level distribution at pixel P. DY1 is the y component 
of the gradient of the gray level distribution at P; G1. G2. ... G8 are the gray levels of pixels P1, P2, ... P8 respectively. 
Thus, on the basis of Formula (5), the gradient of gray level of pixel P can be determined as (-39, -3). 
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[0122] Similarly, the gradients of the gray levels of other pixels in the annular region 300R can be calculated. In the 
present embodiment, the gradients of pixels P1 , P2, P3. P4. P5, P6, 91 and P8 are determined as (-30. -5), (-36, -4). 
(-32, -1). (-33,-6). (-30, -4). (-38. -8). (-33. -3) and (-31, 2) respectively. Thus, the gradient of gray level of each pixel 
In the annular region 300R can be determined in the same way. 
5 [0123] Returning to Fig. 4. in step S43, the first calculating means 212 also determines the weight of the gradient of 
gray level of each pixel in the annular region 300R. For example, the gradient of gray level at a pixel can be given by 
formula (6): 

^0 W1=(|DX1| + tDY1|)/255; (6) 

[0124] Where W1 denotes the weight of a pixel, DX1 is the x component of the gradient of the gray level at the pixel, 
and DY1 is the y component of the gradient of the gray level at the pixel. 

[0125] For example, referred to Fig. 6, In the present embodiment, the gradient of the gray level of pixel P is known 
15 as (-39, -3), hence the weight of the gradient of gray level of pixel P is (|-39| + |-3|)/255=0.165. 

[0126] Likewise, the weights of the gradient of gray level of other pixels P.1. P2. P3, P4. P5, P6, P7. P8. etc., in the 
annular region 300R can be easily determined as 0.137. 0.157, 0.129. 0,153, 0.133, 0.180, 0:141 . 0.129. etc. respec- 
tively. 

20 Calculating the Reference Gradient of Each Pixel in the Annular region 

[0127] Returning to Fig. 4, the flow goes to step S44 after step S43. In step S44. the second calculating means 213 
calculates the gradient of a reference distribution at each point in the above-mentioned annular region 300R. The 
reference distribution represents an ideal model of human face in the region 300A, The gradient of the reference 
25 distribution is to be used by determination means 214 to determine how the portion of inriage in the annular rejgion 
300R is close to a human face image. 

[0128] In the method and apparatus for human face detection of the present invention, human face detection is 
performed by evaluating the difference between the direction of the gradient of the gray level distribution in a relevant 
portion (annular region 300R) of the image to be processed and the direction of a reference gradient derived from the 

30 reference distribution. 

[0129] Specifically, for each sub-image 300A determined, a reference distribution of a human face is determined, 
which can be expressed as: 

wherein h is a constant, a/b equals to the ratio of the width of said sub-image to the height of said sub-image, (x,,. y^.) 
is the center of the sub-image. Figure 10 shows an image with such a reference distribution, and Figure 7 shows the 
40 contour lines of (E11 and E1 2) of the reference distribution. - ' . 

[0130] Then, the gradient of the reference distribution can be expressed as: 

Vz = Oz/ax. 92/ay) = (-2(x-x J/a^. -2(y-y^)/b2) (7) 

45 

[0131] As only the direction of the vector (3z/dx, dzldy) is concerned here, and the reverse direction (-9z/9x, -dz/By) 
will be treated as the same direction in the subsequent steps, proportional changes of the x component and the y 
component of Vz would not change the result of the evaluation. So only the ration a/b Is relievant. A typical value of 
the ratio a/b is 4/5. 

50 [0132] So in step S44, the gradient Vz is calculated using formula (7), and the gradient is called reference gradient. 

[0133] After step S44. the flow goes to step S45. In step S45, for each pixel in the annular region 300R. and angle 

between the gradients Vz and Vg are calculated, where Vg=(DX1. DY1) is the gradient of the gray level distribution. 

Of course. Vg is not limited to being the gradient as calculated using formula (5), rather, it can be calculated using any 

other operator for discrete gradient calculation. 
55 [0134] Specifically, the angle 9 between Vz and Vg at point (x,y) can be calculated using: 

cos e =Vz Vg/(|Vg| |Vz|) 
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e =cos-\| Vz Vg |/{| Vg | | Vz|)) (8) 

where Vz.Vg denotes the Inner product of vectors Vz and Vg. and | Vg | and |Vz | denote the nnagnitudes of 
^ vectors Vg and Vz respectively. Note that e is in the range of 0<G <kI2. 

[0135] Thus, the angle between the two gradients of pixel P is determined as 0.46 (radian). Similarly, the angles 
between the two gradients at other pixels P1 . 92, P3. P4, P5, P6, P7, P8, etc. in the annular region 300R can be easily 
determined as 0.19, 0.26. 0.33, 0.18, 0.23. 0.50. 0.27. 0.42 (radian), etc. respectively 

[0136] Further, In step S45. the average of the angles between the gradient of gray level and Its corresponding 
reference gradient for all pixels In the annular region 300R is given by fomnula (9): 

A1=S1/C1 (9) 

^5 [0137] Where A1 denotes the average of the angles between the gradient of gray level and its corresponding refer- 
ence gradient for all pixels in the annular region, S1 denotes the sum of the angles between the two gradients of all 
pixels in the annular region; C1 denotes the total number of pixels in the annular region. 

[0138] As to the present example, the average angle between the gradient of gray level and its corresponding ref- 
erence gradient of all pixels in the annular region 300R Is determined as 0.59. 
2^ [0139] Returning to step S45 in Fig. 3, after the above-mentioned average of the gradient differences has been 
determined, it is determined in step S45 whether said average is less than a 11th threshold. In the present invention, 
the 11th threshold is between 0.01 and 1 .5, such as 0.61 . If the calculated average is determined to be less than the 
11th threshold, then the flow goes to step S46. If not, the flow goes to step S48. in step S48, it is determined that the 
rectangle 300A does not contain an Image of a human face, and then, the flow ends in step S49. 
[0140] In the present example, the average of the angle between the gradient of gray level and its corresponding 
reference gradient for all the pixels in the annular region 300R is 0.59, which is less than the 11th threshold. Therefore, 
the flow goes to step S46. 

[0141] In step S46. using the weight W1 as described above, a weighted' average of the angle between the gradient 
of gray level and the reference gradient for all the pixels in the annular region 300R is determined, as Is given by 
^ fomnula (10): 

A2=S2/C2; (10) 

^ [0142] Where A2 denotes the weighted average of the angle between the twogradients for all the pixels in the annular 
region, C2 denotes the sum of the weights of pixels in the annular region, S2 denotes the sum of the product of the 
angle and the weight of gradient of gray level for the same pixels in the annular region. 

[0143] In the present example, the weighted average angle between the two gradients for all pixels in the annular 
region is 0.66. 

40 [0144] After the determination of the weighted average angle, it is determined in step S46 whether said weighted 
average angle is less than a 12th threshold, such as 0.68. The 12th threshold in the present invention is ranged from 
0.01 to 1. If the weight average angle is determined to be less than the 12th threshold, then the flow goes to step S47, 
where It is determined that the rectangle 300A contains an image of a human face, then the flow ends in step S49. If 
the weighted average angle is not less than the 12th threshold, then the flow goes to step S48. In step S48. it is 

45 determined that the rectangle 300A does not contain an image of a human face, and then, the flow ends in step S49. 
[0145] As to the present example, the weighted average angle for all pixels in the annular region 300R is 0.66, which 
Is less than the 12th threshold, and then the flow goes to step S47. In step S47. and the region 300A is determined to 
contain an Image of a human face. AftenA/ards, the flow ends in step S49. 

^ The Second Example 

[0146] Flg. SA Is a diagram showing another example of an original image to be detected. The original image 800 
in Fig. 8A is produced by a digital camera. Of course. It also can be input to human face detection system by a digital 
device 111. such as a scanner or the like. Assume that the original image Is stored in a predetermined region in the 
55 HD 1 07 or the RAM 1 09. or the like. 

[0147] Fig. 8B is a diagram showing a rectangular region 800B in the original image 800, which is shown in Fig. 8A, 
and the location in Cartesian coordinate system. In which the X and Y axes extend in horizontal and vertical direction 
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respectively. 

[0148] As to the rectangular region 800B shown In Fig. 8B, the same flow chart shown in Fig. 4 Is adopted to explain 
the process of determining whether the rectangular region 800B contains a human face. 

[0149] Referring to Fig. 4. eye detecting device 218 determines eyes in the image, in step S41, reading means 210 
5 reads the gray level of each pixel in the original image 800. and the sub-image determining device 219 determines 
rectangular regions to be detected, such as rectangular region 8008, in the original image 800. As shown in Fig. 8A» 
the origin of the Cartesian coordinate system is chosen to be at the upper left corner of the original image 800. Then 
the coordinates of each of the four comers of rectangular region BOOB shown in Fig. 8B are (203, 219). (373, 219). 
(203. 423) and (373, 423) respectively in the Cartesian coordinate system. That Is, the width and length of region 8008 
10 are 170 and 204 respectively. 

[0150] Then, in step S42, the annular region 800C around the rectangular region 8008 in Fig. 88 is determined by 
the annular region determination means 211. Referred to Fig. 8C, as to the rectangular region 8008. by scaling its 
width and length with a factor of 9/7 and a factor of 5/7 respectively, two rectangular regions 800C1 and 800C2 are 
generated. 

15 [0151] Fig, 8C is a diagram showing the annular region 800C, which is detemnined for the rectangular region 8008. 
The annular region 800C is formed by the first rectangular region 800C1 and the second rectangular region 800C2. 
The coordinates of each of the four comers of first rectangular region 800C1 in this coordinate system are (170. 190), 
(397, 190), (179, 452) and (397, 452) respectively, while the coordinates of each of the four corners of the second 
rectangular region 800C2 in this coordinate system are (227, 248), (349. 248). (227, 394) and (349, 394) respectively. 

20 . [0152] Then, in step S43, the gradient of the gray level of each pixel in the annular region 800C is determined by 
the first calculating means 212. which is (188, 6). (183. 8), (186, 10), (180, 6). (180, 6), etc. respectively. Similarly, the 
weight of the gradient of the gray level of each pixel in the annular region 800C is determined in the same way, which 
is 0.76, 0.75. 0.77. 0.77, 0.73, etc. respectively 

[0153] Further, in step S44, the reference gradient of each pixel in the annular region 800C is determined respectively 
25 by a second calculating means 213, which is (-0.015, -0.012), (-0.015. -0.012). (-0.015, -0.012), (-0.014, -0.012). etc. 
respectively. Then, the flow goes to step S45. 

[0154] In step S45, the angle between the two gradients of each pixel in the annular region 800C is determined; the 
average angle between the two gradients for pixels in the annular region 800C can be determined in the same way as 
above-mentioned, which is 0.56 for the present example and is less than the 11th threshold. Then the flow goes to 

30 step S46. 

[0155] In step S46, the weighted average angle between the two gradients for pixels in the annular region 800C is 
determined as 0.64. which is less than the 12th threshold, and the flow goes to step S47, where it is determined that 
the rectangular region 800B contains an image of a human face. Then, the flow ends in step S49, 



Alternative Embodiment 

[0156] An alternative embodiment of the present invention is illustrated below. 

[0157] Fig. 9 is a diagram showing the human face determination process according to the alternative embodiment 
of the present invention. In Fig. 9, the same reference numeral as that in Fig. 4 denotes the same process. 
[0158] Again, the original image 800 in Fig. 8A is taken as an example to illustrate determining whether a rectangular 
region 8008 in the original image 800 contains a human face image. 

[0159] First, as in Fig. 4, the process flow starts in step S40. In step S41 , reading means 210 reads the gray level 
of each.pixel in original image 800; eye detecting means 218 detect eye areas in the innage; and sub-image determining 
means 219 detemnines the sub-image(s), such as rectangular region 8008, to be detected based on the eye areas 
detected. Then the flow goes to step S42. In step S42,' an annular region 800C is determined for rectangular region 
8008, as shown in Fig. 8C. Further, the flow goes to step S43, In step S43, the gradient of the gray level at each pixel 
in the annular region 800C is determined, which Is (188, 6). (183. 8). (186, 10), (180. 6), (180. 6); etc. respectively. 
The weight of the gradient of the gray level at each pixel in the annular region 800C is determined in the same way as 
in the previous embodiment, which is 0,76, 0.75, 0.77, 0,77, 0.73, etc. respectively, AftenA/ards, the flow goes to step 
S44. In step S44, the reference gradient of each pixel in the annular region 800C is determined, which is (-0.015. 
-0.012). (-0.015. -0.012), (-0.015. -0.012). (-0.015, -0.012), (-0.014. -0.012). etc. respectively 

[0160] Then, the flow goes to step S95. In step S95, the angle between the gradient of the gray level and Its corre- 
sponding reference gradient at each pixel In the annular region 800C is calculated respectively e.g., in the same way 
as the previous embodiment. Which is 0.64. 0.63. 0.62, 0.64, 0.64, etc. (in radian), respectively Then, the weighted 
average angle between the two gradients for all pixels in the annular region 800C is calculated. While in step S95, it 
is determined whether the weighted average is less than a 13th threshold, such as 0.68. In the present invention, the 
13th threshold is ranged from 0.01 to 1. If the weighted average angle is less than the third threshold, then the flow 
goes to step S96. In step S96, the rectangular region Is determined to contain a human face image. If the weighted 
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average angle is not less than the 13th threshold, then the flow goes to step S97. where it is detemnined that region 
800D does not contain a hunnan face image. Then the flow ends in step S98. 

[0161] For the preset example, since the average of the product of the gradient difference and its corresponding 
gradient weight of each pixel in the annular region is 0.64, which Is less than the third threshold, the flow goes to step 
5 S96 to determine that the rectangular region to be determined 800B contains a human face. Aftenwards, the flow ends 

in step S98. 

[0162] As one skilled in the art may appreciate, it is not necessary that the angle between the gradients of the gray 
level distribution and a reference distribution is calculated and evaluated for every pixel in the annular region; rather, 
the method for detecting human face (or other object to be detected) according to the present invention can be carried 

10 out by calculating and evaluating the angle for only some of the pixels in the annular region. 

[0163] Moreover, although in the above description, a specific embodiment, in which both the average of the angle 
between the gradients of the gray level distribution and a reference distribution and the weighted average of the angle 
are used for evaluation, and another specific embodiment, in which only the weighted average of the angle is used for 
evaluation, are described, an embodiment, in which only the non-weighted average of the angle is used for evaluation. 

15 also can realize the objects present invention and is also included as an embodiment of the present invention. 

[0164] Note that the present invention may be applied to either a system constituted by a plurality of devices (e.g., 
a host computer, an interface device, a reader, a printer, and the like), or an apparatus consisting of a single equipment 
(e.g., a copying machine, a facsimile apparatus, or the like). 

[0165] The objects of the present invention are also achieved by supplying a storage medium. The storage medium 
20 records a program code of a software program that can implement the functions of the above embodiment to the system 
or apparatus, and reading out and executing the program code stored in the storage medium by a computer (or a CPU 
or MPU) of the system or apparatus. In this case, the program code itself read out from the storage medium implements 
the functions of the above-mentioned embodiment, and the storage medium, which stores the program code, consti- 
tutes the present invention. 

25 [0166] As the storage medium for supplying the program code, for example, a floppy disk, hard disk, optical disk, 
magneto-optical disk, CD-ROM, CD-R. magnetic tape, nonvolatile memory card. ROM, and the like may be used. 
[0167] The functions of the above-mentioned embodiment may be implemented not only by executing the readout 
program code by the computer but also by some or all of actual processing operations executed by an OS (operating 
system) running on the computer on the basis of an instruction of the program code. 

30 [01 68] As can be seen from the above, the method of the present invention provides a fast approach for determining 
human face in a picture with a complex background, without requiring the detected picture to have a very high quality, 
thereby substantially eliminating the possibility of the human face in the picture being skipped over. The method allows 
for the precision determination of human face under different scales, orientation and lighting condition. Therefore, in 
accordance with the present invention, with the method, apparatus or system, the human face in a picture can be 

35 quickly and effectively determined. 

[0169] The present invention includes a case where, after the program codes read from the storage medium are 
written in a function expansion card which is inserted into the computer or in a memory provided in a function expansion 
unit which is connected to the computer, CPU or the like contained in the function expansion card or unit performs a 
part or entire process in accordance with designations of the program codes and realizes functions of the above em- 

40 bodiment. 

[0170] (n a case where the present invention is applied to the aforesaid storage medium, the storage medium stores 
programs codes corresponding to the flowcharts (Fig. 4, and 9) described In the embodiments. 
[0171] Although the embodiments of the invention described with reference to the drawings comprise computer 
apparatus and processes performed in computer apparatus, the invention also extends to computer programs on or 

45 in a carrier. The program may be in the form of source or object code or in any other form suitable for use in the 
implementation of the relevant processes. The carrier may be any entity or device capable of carrying the program. 
. [0172] For example, the carrier may comprise a storage medium, such as a ROM, for example a CD ROM or a 
semiconductor ROM. or a magnetic recording medium, for example a floppy disc or hard disc. Further, the carrier may 
be a transmissible carrier such as an electrical or optical signal which may be conveyed via electrical or optical cable 

50 or by radio or other means., 

[0173] When a program is embodied In a signal which may be conveyed directly by a cable or other device or means, 
the carrier may be constituted by such cable or other device or means. 

[0174] Alternatively, the earner may be an integrated circuit in which the program is embedded, the integrated circuit 
being adapted for performing, or for use in the performance of, the relevant processes. 
55 [0175] The embodiment explained above is specialized to determine human face, however, the present invention is 
not limited to determine human face, it is applicable to other determination method, for example, method to detect flaw 
portion on a circuit board. 

[0176] As many apparently widely different embodiments of the present invention can be made without departing 
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from the spirit and scope thereof, it is to be understood that the invention is not limited to the specific embodiments 
thereof except as defined in the appended claims. 



Claims 

1. A method for detecting an object in an image having a gray-level distribution, characterized in that said method 
comprises the steps of: 

a) for a subset of pixels in said image, deriving a first variable from said gray-level distribution of said image, 

b) for said subset of pixels, deriving a second variable from a preset reference distribution, said reference 
distribution being characteristic of said object; 

c) evaluating the correspondence between said first variable and said second variable over the subset of pixels; 
and 

d) determinlng if said Image contains said object based on the result of the evaluation step. 

2. The method of claim 1 , characterized by further comprising the step of: 

selecting an area in said image, 
wherein pixels of said subset are in said area, and 

wherein over pixels in said area the first variable as derived from the gray-level distribution corresponding 
to the object to be detected has good correspondence to the second variable. 

3. The method of claim 2, characterized in that it further comprises the step of: 

determining a sub-image in said image; and 

selecting said area on the basis of the location of said sub-image. 

4. The method of any one of claim 1-3, wherein said first variable represents the direction of the gradient of the gray- 
level distribution of said image. 

5. The method of any one of claim 4. wherein said second variable represents the direction of the gradient of said 
reference distribution. 

6. The method of claim 5. wherein said gradient is a discrete gradient. 

7. The method of claim 5, characterized in that said evaluation step includes a statistical process over said subset 
of pixels. 

8. The method of claim 5, characterized in that said evaluation step includes a statistical processing over said 
subset of pixels and said statistical process comprises a weighting process, in which the weight to each pixel in 
said subset is determined on the basis of the gradient of the gray level distribution at the pixel and pixels with 
larger magnitude of the gradient of the gray level distribution generally have larger weights. 

9. The method of claim 8, characterized in that said evaluating step includes both the statistical process using the 
weighting process and another statistical process without the weighting process and determines whether the image 
contains the object in accordance with the results of both of the statistical processes. 

10. The method of claim 3, characterized in that said area is an annular area, which centers around the center of 

said sub-image. 

11. The method of claim 2. wherein said subset of pixel includes all the pixels in said area. 

12. The method of claim 3, characterized in that said sub-image is detemnined by detecting a predetermined feature 

in said image. 

13. The image of claim 12, characterized in that said predetermined feature includes a pair of dark areas. 
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14. The method of claim 13. characterized in that said sub-image is determined based on the positions of and interval 
between the centers of said pair of dark areas. 

15. The method of claim 13. characterized in that said reference distribution is expressed by 

5 

z(x.y)= -(x^/a^+y^/t>^)+h 

wherein h is a constant, a/b equals to the ratio of the width of said sub-image to the height of said sub-image. 
'to and the origin is at the center of said sub-image. 

16. The method of claim 5 or 15, wherein said evaluating step includes: 

for all the pixels in said subset, calculating the average value of the angle between the gradient of the gray- 
^5 level distribution of said image and the gradient of said reference distribution; 

comparing said average value with a predetermined value; and 

determining that the sub-image contains said object if said average value equals to or is smaller than said 
predetermined value. 

20 17. The method of claim 5 or 15, wherein said evaluating step includes: 

for all the pixels in said subset, calculating a weighted average value of the angle between the gradient of the 
gray-level distribution of said image and the gradient of said reference distribution, wherein a pixel having a 
greater magnitude of gradient of gray-level distribution generally has a larger weight; 
25 comparing said weighted average value with a predetermined value; and 

determining that the sub-image contains said object if said weighted average value equals to or is smaller 
than said predetermined value. 

18. The method of claim 5 or 15. wherein said evaluating step includes: 

30 

for all the pixels in said subset, calculating the average value of the angle between the gradient of the gray- 
level distribution of said Image and the gradient of said reference distribution; 

for all the pixels in said subset, calculating a weighted average value of the angle between the gradient of the 
gray-level distribution of said image and the gradient of said reference distribution, wherein a pixel having a 
35 greater magnitude of gradient of gray-level distribution has a larger weight; 

comparing said average value with a first predetermined value; 
comparing said weighted average value with a second predetermined value; and 

determining that the sub-image contains said object if said average value equals to or is smaller than said first 
predetermined value and said weighted average value equals to or is smaller than said second predeterniined 
^0 value. 

19. A method for detecting an object in an image having a gray-level distribution, characterized in that said method 
comprises the steps of: 

45 a) determining a sub-image in said image; 

b) selecting a subset of pixels in said image based on said sub-image; 

c) for pixels of said subset, deriving a first variable from said gray-level distribution of said Image; 

d) for said subset of pixels, deriving a second variable from a preset reference distribution, said reference 
distribution being characteristic of said object; 

50 e) evaluating the correspondence between said first variable and said second variable over the subset of 

pixels; and 

f) determining if said image contains said object based on the result of the evaluation step. 

20. A storage medium with a program code stored therein for detecting an' object in an image with a gray-level distri- 
55 bution. characterized in that said program code comprises: 

codes for determining a sub-Image in said image; 

codes for deriving, for a subset of pixels, a first variable from said gray-level distribution of said image. 
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codes for deriving, for sard subset of pixels, a second variable from a preset reference distribution, sard ref- 
erence distribution being characteristic of said object; 

codes for evaluating the con-espondence between said first variable and said second variable over the subset 
of pixels; and 

5 codes for determining if said image contains sard object based on the result of the evaluation step. 

21. An image processing method for determining a feature portion in an image, comprising: 

a reading step of reading the image and a rectangle region to be determined in the image; 
fo a setting step of setting a annular region surrounding said rectangle region; 

a step of calculating the gradient of gray level of each pixel in said annular region; 

a step of determining a reference distribution for the annular region and determining a annular region reference 
gradient of each pixel In the annular region; and 

a determination step of determining the feature portion contained In said rectangle region on the basis of the 
15 gradient of the gray level of each pixel and the reference gradient of each pixel in said annular region. 

22. The image processing method according to claim 21, wherein the image containing said rectangle region to be 
determined is a feature portion. 

20 23. The image processing method according to claim 21, wherein sard annular region is formed by a first rectangle 
region and a second rectangle region. 

24. The image processing method according to claim 23, wherein said first rectangle region is located inside said 
rectangle region to be determined. 

25 

25. The image processing method according to claim 23, wherein said second rectangle region is located outside said 
rectangle region to be determined. 

26. The image processing method according to claim 23, wherein the center of siaid first rectangle region lies in sub- 
30 stantially the same location as that of said rectangle region to be determined, and the width and the length of said 

first rectangle region respectively equals m times the width and the length of said rectangle region to be determined, 
where 0 < m < 1 . 

27. The image processing method according to claim 23, wherein the center of said second rectangle region lies in 
35 the same location as that of said rectangle region to be determined, and the width and the length of said second 

rectangle region respectively equals n times the width and the length of said rectangle region to be determined, 
where 1 < n < 2. 

28. The image processing method according to claim 21, wherein said step of calculating the gradient of gray level 
40 comprising the step of calculating the gradient of the gray level of a pixel on the basis of the gray level of the pixels 

surrounding the pixel. 

29. The image processing method according to claim 28, wherein the number of pixels surrounding said pixel equals 
k2, where k is an Integer, and 2 < k < 15. 

45 

30. The image processing method according to claim 21, wherein reference distribution Is determined in accordance 
to the rectangle region. 

31. The image processing method according to claim 21 or 30, further comprising the step of determining the angle 
50 the gradient of the gray level and the reference gradient of each pixel In said annular region. 

32. The image processing method according to claim 21 or 30, further comprising the step of determining the average 
of the angle between the gradient of the gray level of each pixel and the reference gradient of each pixel In said 
annular region. 



55 



33. The image processing method according to claim 21 or 32, further comprising the step of determining the weight 
of the gradient of the gray level of each pixel in said annular region. 
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34. The Image processing method according to claim 32, further comprising the step of determining whether the av- 
erage of the angle between the gradient of the gray level of each pixel and the reference gradient of each pixel in 
said annular region is less than a first threshold, where the first threshold is within the range of 0.01 to 1 .5, preferably 
of 0.61. 

5 

35. The image processing method according to claim 34, further comprising the step of determining whether the weight- 
ed average of the angle between the gradient of the gray level of each pixel and the reference gradient of each 
pixel in said annular region with the weight of the gradient of the gray level of each pixel in said annular region is 
less than a second threshold, where the second threshold is within the range of 0.01 to 1 , preferably of 0.68. 

10 

36- The image processing method according to claim 33, further comprising the step of determining whether the weight- 
ed average of the angle between the gradient of the gray level of each pixel and the reference gradient with the 
weight of the gradient of the gray level of each pixel in said annular region is less than a third threshold, where the 
third threshold is within the range of 0.1 to 1, preferably of 0.68. 

15 

37. A human eye detecting method for detecting human eye in an image, comprising: 

a reading step of reading the gray level of each pixel in the each column in the image; 
a segmenting step of segmenting each column Into a plurality of Intervals , and labeling each of the Inten^als 
20 as valley region , intermediate region or peak region; 

a merging step of merging the valley region of the each column and the valley region of its adjacent column, 
and generating an eye candidate region; and 

a determining step of determining the human eye from the eye candidate regions. 

25 38. The method according to claim 37, wherein the segmenting step includes the step of labeling each interval as one 
of valley region, intermediate region and peak region based on the gray level of each the interval in a column. 

39. The method according to claim 37, wherein the segmenting step includes the step of labeling each interval as one 
of valley region, intermediate region and peak region based on the luminance value of each interval In a column. 

30 

40. The method according to claim 37, wherein the segmenting step includes the step of labeling each Interval In a 
column as one of valley region, intermediate region and peak region based on the respective gray level ratios of 
the interval to its two adjacent intervals. 

35 41. The method according to claim 37, wherein the segmenting step includes the step of labeling each interval In a 
column as one of valley region, intermediate region and peak region based on the respective luminance value 
ratios of the Interval to Its two adjacent Interval. 

42. The method according to any one of claims 37, further comprising the step of comparing the gray level of a segment 
40 point with that of its two adjacent intervals, and adding the segment point Into the adjacent Interval which has a 

gray level close to that of the segment point. 

43. The method according to any one of claims 37, further comprising the step of comparing the gray level of a inter- 
mediate region with that of Its adjacent valley region and peak valley, and adding the intermediate region Into the 

45 valley region on the basis of the gray levels of the adjacent valley region, peak region and the Intermediate region. 

44. The method according to claim 37, wherein the merging step comprising the steps of 

setting each valley region In the first column of the image as a seed region respectively; 
50 determining whether the valley region of next column of the image can be merged Into the seed region; 

merging the valley region that can be merged into the seed region; 
setting the non-merged valley region as a seed region; and 

determining the seed region which has no further merged valley region as an eye candidate region. 

55 45. The method according to claim 44, further comprising the step of comparing the size of overlap of the valley region 
and the seed region's predicted valley region, the valley region and the seed region's predicted valley region, 

46. The method according to claim 44, further comprising the step of comparing the gray level of the valley region and 
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that of the seed reglon. 

47. The method according to claim 44, further comprising the step of comparing the gray levels of the valley region, 
the seed region, the background gray levels of the valley region and the seed region. 

48. The method according to claim 37, wherein the determining step comprising the step of determining whether the 
gray level of the eye candidate region is less than the first threshold, where the first threshold is within the range 
of 100 to 200. preferably of 160. 

49. The method according to claim 48, wherein the determining step further comprising the step of determining whether 
the background gray level of the eye candidate region is bigger than the second threshold, where the second 
threshold is within the range of 20 to 80, preferably of 30. 

50. The method according to claim 49, wherein the determining step further comprising the step of determining whether 
the difference of background gray level of the eye candidate region and its gray level is bigger than a third threshold, 
where the third threshold is within the range of 5 to 120 , preferably of 20. 

51 . The method according to claim 50, wherein the determining step further comprising the step of determining whether 
the ratio of the width to the height of an eye candidate region Is bigger than a fourth threshold, where the fourth 
threshold is within the range of 1 to 5, preferably of 3.33. 

52. The method according to claim 51 , wherein the determining step further comprising the step of determining whether 
the ratio of the size of an eye candidate region to that of its circum-rectangle is bigger than the fifth threshold, 
where the fifth threshold is within the range of 0.2 to 1 preferably of 0.4. 

53. The method according to claim 14, wherein the step for determining a sub-image in said image includes: 

segmenting each column of the image into a plurality of intervals , 

labeling each of the intervals as valley region , intermediate region or peak region; 

merging the valley region of the each column and the valley region of its adjacent column, thereby generating 
a candidate dark area; and 
determining a dark -area. 

54. The method according to claim 53, wherein the merging step comprising the steps of 

setting each valley region in the first column of the image as a seed region; 

determining whether the valley region of the next column of the image can be merged into the seed region, 
merging the valley region that can be merged into the seed region; 

setting the valley region that cannot be merged into the seed region as a seed region; and 
determining the seed region which has no further merged valley region as a dark area. 

55. Apparatus for detecting an object in an image having a gray-level distribution, comprising: 

a) means for deriving a first variable from said gray-level distribution of said image for a subset of pixels in 

said image, 

b) means for deriving a second variable from a preset reference distribution for said subset of pixels, said 
reference distribution being characteristic of said object; 

c) correspondence evaluating means for evaluating the correspondence between said first variable and said 
second variable over the subset of pixels; and 

d) means for determining if said image contains said object based on the processing by the correspondence 
evaluating means. 

56. The apparatus of claim 55, further comprising means for selecting an area in said image, and wherein: 

pixels of said subset are in said area, and 

over pixels in said area, the first variable as derived from the gray-level distribution corresponding to the object 
to be detected has good correspondence to the second variable. 
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57. The apparatus of claim 56, further comprising: 

means for determining a sub-image in said image; and 

means for selecting said area on the basis of the location of said sub-image. 

5 

58. The apparatus of any one of claim 55-57, wherein said first variable represents the direction of the gradient of the 
gray-level distribution of said image. 

59. The apparatus of claim 58. wherein said second variable represents the direction of the gradient of said reference 
10 distribution. 

60. The apparatus of claim 59. wherein said gradient is a discrete gradient. 

61. The apparatus of claim 59, wherein the correspondence evaluating means is arranged to perform a statistical 
15 process over said subset of pixels. 

62. The apparatus of claim 59, wherein the correspondence evaluating means is arranged to perform statistical 
processing over said subset of pixels and said statistical process comprises a weighting process, in which a weight 
to each pixel in said subset is determined on the basis of the gradient of the gray level distribution at the pixel and 

20 pixels with larger magnitude of the gradient of the gray level distribution generally have larger weights. 

63. The apparatus of claim 62, wherein the correspondence evaluating means is arranged to perform processing that 
includes both the statistical process using the weighting process and another statistical process without the weight- 
ing process, and is arranged to determine whether the image contains the object in accordance with the results 

25 of both of the statistical processes. 

64. The apparatus of claim 57, wherein said area is an annulararea, which centers around the center of said sub-image. 

65. The apparatus of claim 56, wherein said subset of pixel includes all the pixels in said area. 

30 

66. The apparatus of claim 57, wherein the means for determining a sub-image is arranged to determine said sub- 
image by detecting a predetermined feature in said image. 

67. The apparatus of claim 66, wherein said predetermined feature includes a pair of dark areas. 

35 

68. The apparatus of claim 67, wherein the means for detecting a sub-image is arranged to determine said sub-image 
based on the positions of and interval between the centers of said pair of dark areas. 

69. The apparatus of claim 67, wherein said reference distribution is expressed by 

40 

z(x.y)= -{x^/a^V/b^)+h 

wherein h Is a constant, a/b is the ratio of the width of said sub-image to the height of said sub-Image, and the 
45 origin is at the center of said sub-image. 

70. The apparatus of claim 59 or 69, wherein said correspondence evaluating means comprises: 

means for calculating, for all the pixels In said subset, the average value of the angle between the gradient of 
50 the gray-level distribution of said image and the gradient of said reference distribution; and 

means for comparing said average value with a predetermined value; 

and wherein the means for determining if the image contains the object is arranged to determine that the sut)- 
image contains said object if said average value is equal to or is smaller than said predetermined value. 

55 71. The apparatus of claim 59 or 69, wherein said correspondence evaluating means comprises: 

means for calculating, for all the pixels in said subset, a weighted average value of the angle between the 
gradient of the gray-level distribution of said image and the gradient of said reference distribution, wherein a 
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pixel having a greater magnitude of gradient of gray-level distribution generally has a larger weight; and 
means for comparing sard weighted average value with a predetermined value;. 

and wherein the means for determining if the image contains the object is arranged to determine that the sub- 
image contains said object if said weighted average value is equal to or is smaller than said predetemnined 
value. 

72. The apparatus of claim 59 or 69. wherein said correspondence evaluating means comprises: 

means for calculating, for all the pixels in said subset, the average value of the angle between the gradient of 

the gray-level distribution of said image and the gradient of said reference distribution; 
means for calculating, for all the pixels in said subset, a weighted average value of the angle between the 
gradient of the gray-level distribution of said image and the gradient of said reference distribution, wherein a 
pixel having a greater magnitude of gradient of gray-level distribution has a larger weight; 
means for comparing said average value with a first predetermined value; and 
means for comparing said weighted average value with a second predetermined value; 
and wherein the means for determining if the image contains the object is arranged to determine that the sub- 
image contains said object if said average value is equal to or is smaller than said first predetermined value 
and said weighted average value is equal to or is smaller than said second predetermined value. 

73. Apparatus for detecting an object in an image having a gray-level distribution, comprising: 

a) means for determining a sub-image in said image; 

b) means for selecting a subset of pixels in said image based on said sub-image; 

c) means for deriving, for pixels of said subset, a first variable from said gray-level distribution of said Image; 

d) means for deriving, for said subset of pixels, a second variable from a preset reference distribution, said 
reference distribution being characteristic of said object; 

e) evaluating means for evaluating the correspondence between said first variable and said second variable 
over the subset of pixels; and 

f) means for determining if said image contains said object based on the result of the processing by the eval- 
uating means. 

74. An image processing apparatus for determining a feature portion in an image, comprising: 

reading means for reading the image and a rectangle region to be determined in the image; 
setting means for setting an annular region surrounding said rectangle region; 

gradient calculating means for calculating the gradient of gray level of each pixel in said annular region; 

reference distribution determining means for determining a reference distribution for the annular region and 

determining an annular region reference gradient of each pixel in the annular region; and 

feature portion determination means for determining the feature portion contained in said rectangle region on 

the basis of the gradient of the gray level of each pixel and the reference gradient of each pixel in said annular 

region. 

75. The apparatus according to claim 74, wherein the image containing said rectangle region to be determined is a 
feature portion. 

76. The apparatus according to claim 74, wherein the setting means is arranged to form said annular region by a first 
rectangle region and a second rectangle region. 

77. The apparatus according to claim 76. wherein the setting means is arranged to locate said first rectangle region 
inside said rectangle region to be determined. 

78. The apparatus according to claim 76, wherein the setting means is arranged to locate said second rectangle region 
outside said rectangle region to be determined. 

79. The apparatus according to claim 76, wherein the setting means is arranged to locate the center of said first 
rectangle region in substantially the same location as that of said rectangle region to be determined, and to set 
the first rectangke region so that the width and the length of said first rectangle region respectively equals m times 
the width and the length of said rectangle region to be determined, where 0 < m < 1. 
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80. The apparatus according to claim 76. wherein the setting means is arranged to locate the center of said second 
rectangle region in the same location as that of said rectangle region to be determined, and to set the second 
rectangle region so that the width and the length of said second rectangle region respectively equals n times the 
width and the length of said rectangle region to be determined, where 1 < n < 2. 

5 

81. The apparatus according to claim 74, wherein said gradient calculating means comprises means for calculating 
the gradient of the gray level of a pixel on the basis of the gray level of the pixels surrounding the pixel. 

82. The apparatus according to claim 81 , wherein the gradient calculating means is arranged to select the number of 
10 pixels surrounding said pixel to be used in the gradient calculation to be k^, where k is an integer, and 2 < k < 15. 

83. The apparatus according to claim 74, wherein the reference distribution determining means is arranged to deter- 
mine said reference distribution in accordance with the rectangle region. 

15 84. The apparatus according to claim 74 or 83, further comprising means for determining the angle of the gradient of 
the gray level and the reference gradient of each pixel in said annular region. 

85. The apparatus according to claim 74 or 83. further comprising means for determining the average of the angle 
between the gradient of the gray level of each pixel and the reference gradient of each pixel in said annular region. 

20 

86. The apparatus according to claim 74 or 85, further comprising means for determining the weight of the gradient 
of the gray level of each pixel in said annular region. 

87. The apparatus according to claim 85. further comprising means for determining whether the average of the angle 
25 between the gradient of the gray level of each pixel and the reference gradient of each pixel in said annular region 

is less than a first threshold, where the first threshold is within the range of 0.01 to 1.5. 

88. The apparatus according to claim 87, wherein the first threshold is 0.61. 

30 89. The apparatus according to claim 87 or claim 88, further comprising means for determining whether the weighted 
average of the angle between the gradient of the gray level of each pixel and the reference gradient of each pixel 
in said annular region with the weight of the gradient of the gray level of each pixel in said annular region is less 
than a second threshold, where the second threshold is within the range of 0.01 to 1. 

35 90. The apparatus according to claim 89, wherein the second threshold is 0.68. 

91. The apparatus according to claim 86, further comprising means for determining whether the weighted average of 
the angle between the gradient of the gray level of each pixel and the reference gradient with the weight of the 
gradient of the gray level of each pixel in said annular region is less than a third threshold, where the third threshold 

40 is within the range of 0.1 to 1 . 

92. The apparatus according to claim 91 , wherein the third threshold is 0.68. 

93. Apparatus for detecting a human eye in an image, comprising: 

45 

reading means for reading the gray level of each pixel in the each column in the image; 

segmenting means for segmenting each column into a plurality of intervals, and labeling each of the Intervals 

as valley region, intermediate region or peak region; 

merging means for merging the valley region of the each column and the valley region of Its adjacent column. 

50 and generating an eye candidate region; and 

determining means for determining if a human eye exists in the eye candidate regions. 

94. The apparatus according to claim 93, wherein the segmenting means includes means for labeling each interval 
as one of valley region, intermediate region and peak region based on the gray level of each interval in a column. 



55 



95. The apparatus according to claim 93, wherein the segmenting means includes means for labeling each interval 
as one of valley region, intermediate region and peak region based on the luminance value of each intense! In a 
column. 
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96. The apparatus according to claim 93, wherein the segmenting means includes means for labeling each interval in 
a column as one of valley region, intermediate region and peak region based on the respective gray level ratios 
of the interval to its two adjacent intervals. 

5 97. The apparatus according to claim 93, wherein the segmenting means includes means for labeling each interval in 
a column as one of valley region, intermediate region and peak region based on the respective luminance value 
ratios of the interval to its two adjacent intervals. 

98. The apparatus according to claim 93, further comprising means for comparing the gray level of a segment point 
10 with that of its two adjacent intervals, and adding the segment point into the adjacent interval which has a gray 

level close to that of the segment point. 

99. The apparatus according to claim 93, further comprising means for comparing the gray level of an intermediate 
region with that of its adjacent valley region and peak valley, and adding the intermediate region into the valley 

15 region on the basis of the gray levels of the adjacent valley region, peak region and the intermediate region. 

100. The apparatus according to claim 93. wherein the merging means comprises: 

means for setting each valley region in the first column of the image as a seed region respectively; 
20 means for determining whether the valley region of a next column of the image can be merged into the seed 

region; 

means for merging a valley region that can be merged into the seed region; 
means for setting a non-merged valley region as a seed region; and 

means for determining the seed region which has no further merged valley region as an eye candidate region. 

25 

101. The apparatus according to claim 100, further comprising means for comparing the size of overlap of the valley 
region and the seed region's predicted valley region. 

102. The apparatus according to claim 1 00, further comprising means for comparing the gray level of the valley region 
30 and that of the seed region. 

lOS.The apparatus according to claim 100. further comprising means for comparing the gray levels of the valley region, 
the seed region, the background gray levels of the valley region and the seed region. 

35 104.The apparatus according to claim 93, wherein the determining means comprises means for determining whether • 
the gray level of the eye candidate region is less than a first threshold, where the first threshold is within the range 
of 100 to 200. 



lOS.The apparatus according to claim 104, wherein the first threshold is 160. 

40 

lOe.The apparatus according to claim 104 or claim 105, wherein the determining means further comprises means for 
detemriining whether the background gray level of the eye candidate region is bigger than a second threshold, 
where the second threshold is within the range 20 to 80. 

45 107.The apparatus according to claim 106, wherein the second threshold is 30. 

lOS.The apparatus according to claim 106 or claim 107. wherein the determining means further comprises means for 
determining whether the difference of background gray level of the eye candidate region and its gray level is bigger 
than a third threshold, where the third threshold is within the range 5 to 120. 

50 

109. The apparatus according to claim 108, wherein the third threshold is 20. 

110. The apparatus according to claim 108 or claim 109. wherein the determining means further comprises means for 
determining whether the ratio of the width to the height of an eye candidate region is bigger than a fourth threshold, 

55 where the fourth threshold is within the range of 1 to 5. 

m.The apparatus according to claim 110. wherein the fourth threshold is 3.33. 
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112.The apparatus according to claim 110 or claim 111 , wherein the determining means further comprises means for 
determining whether the ratio of the size of an eye candidate region to that of its circum-rectangle is bigger than 
a fifth threshold, where the fifth threshold is within the range of 0.2 to 1 . 

5 113.The apparatus according to claim 112, wherein the fifth threshold is 0.4. 

114.The apparatus according to claim 68, wherein the means for detennlning a sub-image in said image includes: 

means for segmenting each column of the image into a plurality of intervals, 
10 means for labeling each of the intervals as valley region, intermediate region or peak region; 

means for merging the valley region of each column and the valley region of its adjacent column, thereby 
generating a candidate dark area; and 
means for determining a dark area. 

15 liS.The apparatus according to claim 114. wherein the merging means comprises: 

means for setting each valley region in the first column of the image as a seed region; 

means for determining whether the valley region of the next column of the image can be merged into the seed 

region, 

20 means for merging the valley region that can be merged into the seed region; and 

means for setting the valley region that cannot be merged into the seed region as a seed region; 

and wherein the means for determining a dark area is arranged to determine a seed region which has no 

further merged valley region as a dark area. 

25 116.A storage device storing instructions for causing a programmable processing apparatus to become operable to 
perform a method as set out in at least one of claims 1 to 19, and 21 to 54. 

117.A signal carrying instructions for causing a programmable processing apparatus to become operable to perform 
a method as set out in at least one of claims 1 to 19. and 21 to 54. 
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